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ABSTRACT 

Results of administering a computerized adaptive 
version of the Grammar, Spelling, and Punctuation Test (GS?) to 
students in the College of Communication at the University of Texas, 
Austin were studied. The computerized adaptive version (CAT) was 
administered for the first time in June 1992 to 35 prospective 
students who participated in the College of Communication's 
Minorities in Communication (MICOM) program. MICOM is a minority 
student recruitment program for high-achieving minority students. 
Statistical comparisons of the two versions conducted in a previous 
study were used to indicate the relationship between the 
paper-and-penci 1 scores and CAT scores. One result of the test 
administration is the average total number of items administered on 
the computer version of the test was 60.5 items, as opposed to the 
100 items required on the paper-and-penc i 1 version. MICOM students 
taking the CAT version were asked for feedback about the test and 
their experiences with mi crocomput ers . S t udent s had f avorabl e 
attitudes about the CAT test, and most indicated that they would 
prefer it to the paper-and-penci 1 version. Students also indicated a 
great deal of experience with computers. One table summarizes study 
findings. (SLD) 
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The Grammar, Spelling, and Punctuation (GSP) test is a placement test administered to 
students in the College of Communication as a means of determining eligibility to register for 
certain courses in Journalism, Advertising, and Radio-TV-Film. The 100-question 
examination is offered at the beginning of each semester and prior to pre-registration periods 
during the fall and spring semesters. Students are administered one of six forms of the 
multiple-choice test in typical paper-and-pencil format. 

In the field of psychometrics, an increasing amount of research is being conducted in the 
area of computerized adaptive testing (CAT). Computerized adaptive testing procedures 
involve the repeated estimation of an examinee's ability level based on his or her responses to 
items which have been presented and the adaptation of further item selection according to each 
item's degree of difficulty and the examinee*s latest ability estimate. Advantages of 
computerized adaptive testing over paper-and-pencil versions include administration of only 
those items appropriate for an examinee's ability level, shorter test length (usually one-fourth to 
one-third shorter) with equal or greater precision of measurement, and immediate feedback 
provided to the examinee. The purpose of this study was to investigate the results of 
administering a computerized adaptive version of the GSP as well as to gain insight into 
examinees' attitudes towards this particular format of the test 

The computerized adaptive version of the GSP was administered for the first time by the 
staff of the Measurement and Evaluation Center on June 24, 1992, to prospective students who 
participated in the College of Communication's Minorities in Communication (MICOM) 
program. 
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Methcxi 

Subjects 

The MICOM program in the College of Coirimunication is a minority student recruitment 
effort for high-achieving minority students from Texas high schools. These students are 
brought to the University of Texas at Austin for five days of structured activities during the 
summer between their junior and senior years. One of the scheduled activities for the 1992 
group was to take the computerized version of the GSP test. The MICOM group consisted of 
35 minority students. 

Materials 

The item bank for the CAT version was developed from the pool of items contained in the 
six forms of the paper-and-pencil version. The items span a wide range of difficulty levels. 

Examinees may take the sections in any order on the CAT version; however, they cannot 
move between sections. Each section must be completed before moving on to the next one. 
Within each section, items are selected and presented in a tailored fashion based upon each 
examinee^s performance on previously administered items. Specifically, the first item 
administered to each examinee is randomly selected from those items which have a difficulty 
level in the middle range. If the examinee responds correctly to the first item, the next item 
presented will be more difficult than the first; however, if the ey^aminee responds incorrectly, 
an easier item is presented next. This process of adapting the presentation of items to match the 
examinee's skill level based on responses to the previous items is continued until a sufficiently 
accurate estimate of the student's skill is obtained or a predetermined maximum number of 
items have been administered within a section. The result is that students are administered test 
items that are appropriate for their skill level; consequently, they are not administered items that 
are either too easy or too difficult for them. For this reason, students are not typically required 
to take as many test items with the computer adaptive version as they are with the paper-and- 
pencil version. For this study, the prespecified maximum number of items per section 
corresponded with the limits on the paper-and-pencil version: 33 items for the Grammar and 
Punctuation sections and 34 items for the Spelling section. 

The computer adaptive and paper-and-penci! vef^^ions of the GSP test are reported on 
different scales. Scores on the paper-and-pencil ve^sk n are reported on the traditional number 
correct scale, whereas scores on the computer adaptive version are reported on a 20 to 80 scale. 
A score of 72 is considered to be passing on the paper-and-pencil version. Statistical 



ERLC 



5 



3 



comparisons of the two versions of the test conducted \n a previous study indicate that a 
composite score of 45 on the computer adaptive version xs equivalent to a score of 72 on the 
paper-and-pencil version. The composite score is the average of the Grammar, Spelling, and 
Punctuation scale scores on the CAT version. 

Pr ocedure 

In 1991, a study was conducted to calibrate the items from the six paper- and-pencil forms 
of the GSP in order to develop a large item bank for a computerized adaptive version of the 
test, though not all items from the paper-and-pencil versions were included in the final CAT 
item bank. 

On June 24, 1992, 35 students in the MICOM group in the College of Communication 
were administered the CAT version of the GSP. Examinees were presented with instructions 
via computer and allowed time to become familiar with the keyboard and the appropriate 
method of responding. As stated previously, examinees were able to select the order in which 
they took the three sections, but they were required to complete each section before moving on 
to the next one. Results were presented to the student through the computer immediately after 
completion of the three sections. 

After the students completed the computer adaptive version of the GSP test, they were 
asked to provide feedback about the test and about their previous experiences with 
microcomputers by responding to a set of 1 1 questions. These questions were also presented 
via the computer. 

Results of the Test Administiution 

The average time taken from the presentation of the first screen of instructions to 
completion of the last of the three sections of the GSP test was 39.4 minutes, whereas 
examinees are allowed up to 90 minutes to complete the paper-and-pencil version of the test. 
The average time taken for each of the three sections was 15.4 minutes for the Grammar 
section, 4.4 minutes for the Spelling section, and 14.5 minutes for the Punctuation section. 
These means, along with the corresponding standard deviations and minimum and maximum 
times, are presented in Table 1. 

The papcr-and-pencil version of the GSP contains 100 items (33 in the Grammar section, 
34 in Spelling, and 33 in Punctuation), and examinees are required to take all items; however, 
examinees are not required to take 100 items on the computerized version. The test ends when 
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an accurate estimate of one's ability has been obtained or all items in a section have been 
administered. The average number of test items administered for each section was 19.6 items 
for the Grammar section, 21.7 for Spelling, and 19.2 for Punctuation. Overall, the average 
total number of items administered on the computer version was 60.5 items, as opposed to the 
100 items required on the paper-and-pencil version. Table 1 presents the mean number of 
items administered per section as well as additional descriptive statistics. 

The average scaled scores (20 to 80 scale) were 42.5 for Grammar, 50.5 for Spelling, and 
45.3 for Punctuation. The average composite score was 46.1. Sixty percent of the students 
obtained a score of 45 or greater. These means and additional descriptive statistics are shown 
in Table 1. 

Table 1 

Descriptive Statistics for the Test Scores, Number of Items Administered, 
and Testing Times from the Computerized Version of the 
Gnimmar, Spelling, and Punctuation Test Given 
to the MICOM Group on June 24, 1992 
(N=35) 



Scale 


Mean 


SD 


Minimum 


Maximum 


Composite 


46.09 


8.32 


32.00 


61.00 


NIA 


60.51 


7.70 


53.00 


85.00 


TT 


34.30 


7.04 


18.70 


51.72 


Grammar 


42.54 


11.68 


20.00 


66.00 


NIA 


19.60 


2.77 


17.00 


33.00 


TT 


15.41 


4.52 


7.45 


30.22 


Spelling 


50.49 


9.16 


31.00 


69.00 


NIA 


21.74 


5.03 


17.00 


34.00 


TT 


4.42 


1.54 


1.77 


9.17 


Punctuation 


45.26 


10.43 


20.00 


60.00 


NIA 


19.17 


2.06 


17.00 


26.00 


TT 


14.47 


3.50 


7.42 


22.83 



Note . NIA is the number of items administered and TT is actual testing time. 
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Student Feedback 

After the students completed the computer adaptive GSP test, they were presented 1 1 
questions conceming their attitudes about the computerized test and their previous experience 
witi computers. These questions and the distribution of responses to each question are shown 
below. 

The directions for the computerized version were clear. 

Strongly Agree 54.3% 

Agree 45.7% 

Neutral 0.0% 

Disagree 0.0% 

Strongly Disagree 0.0% 

The arrow keys used to enter my answers were easy to use. 

Strongly Agree 68.6% 

Agree 22.9% 

Neutral 2.9% 

Disagree 5.7% 

Strongly Disagree 0.0% 

I liked getting my scores immediately after the test was finished. 

Strongly Agree 85.7% 

Agree 11.4% 

Neutral 2.9% 

Disagree 0.0% 

Strongly Disagree 0.0% 

I think that the computerized version assessed my skills accurately. 

Strongly Agree 37.1% 

Agree 25.7% 

Neutral 28.6% 

Disagree 2.9% 

Strongly Disagree 5 .7 % 
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Which version of the GSP would you prefer to take? 

Computerized 68.6% 

Paper-and-Pencil 22.9% 

No preference 8.6% 



Have you ever used a computer at school? 

Yes 94.3% 

No 5.7% 

Have you ever used a personal computer outside of school? 

Yes 80.0% 

No 20.0% 

Have you used a personal computer to play games? 

Yes 94.3% 

No 5.7% 

Were any of your lessons in school presented on a computer? 

Yes 62.9% 

No 37.1% 

Have you used a computer for word processing? 

Yes 88.6% 

No 11.4% 



Were any of the tests you took in school given on a computer? 

Yes 31.4% 
No 68.6% 



It is evident from the students' responses to these questions that theu- attitudes about the 
computer adaptive version of the GSP test were quite favorable. The majority of the students 
indicated that they would prefer to take the computerized version rather than the paper-and- 
pencil version. The students also indicated a great deal of previous experience with computers. 
It is interesting to note that the two questions with the fewest number of affirmative responses 
were the ones concerning the presentation of school lessons via computer and computerized 
testing (adaptive or not) in school. 
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